Neural Networks in Python from Scratch: Learning by Doing

Discover how interpolation guides neural networks, defines and minimizes error to fit data, and how neurons, weights, bias, and nonlinear activations enable learning inspired by the brain.

Explore how neural networks process inputs by multiplying pixel values with weights to form hidden activations and outputs, and train by minimizing error to classify age groups.

Build a simple two-input, one-output neural network to learn the sum of two numbers, using random weights and an optional bias, trained in Python with a provided notebook.

Prepare data for a neural network that learns addition by generating -20 to 20 input pairs with a bias and splitting into train and test; targets are sums.

Initialize random weights in -2 to 2 for a simple one-input, one-output neural network; compute the output via dot product; train toward adding two numbers with weights w0=0, w1=1, w2=1.

Define accuracy and error functions for a neural network, show calculating outputs for all inputs, and explain training with gradient descent using training and testing data.

Compute the gradient of the error function to adjust three neural network weights, using the chain rule, and compare slow versus fast gradient implementations in Python.

train a neural network from scratch using gradient descent, updating weights with a learning rate to minimize error and converge toward the target weights 0, 1, and 1.

Train a neural network on 700 data points to learn addition, then test on 300 unseen examples, achieving 100% accuracy, and explore learning rate and iterations to converge.

Extend a first neural network to classify the sign of the sum of two numbers using a hidden layer, two outputs, and a sigmoid activation in Python.

Modify input, output, and weights to enable a two-output network that signs the sum, adding a hidden layer and dedicated negative/positive outputs.

Adapt the accuracy function for vector outputs using arc max, and derive gradients for a two-output network via the chain rule to enable gradient descent.

Define hidden values, dot products, and derivatives to compute gradients of the error function with respect to weights in neural networks, and optimize with vectorized operations for faster training.

train a modified neural network by adjusting iterations, not storing weights, and updating gradients with calculate next for hidden and output values to enable number recognition in the next lecture.

Learn how a neural network with a hidden layer and ten output neurons recognizes handwritten digits from image pixels, training with data, calculating gradients, and testing for accuracy.

Apply a neural network to handwritten digit recognition using 28x28 images from 60,000 training samples and ten-digit output vectors, with training and test data from four files.

Speed up neural network training by replacing slow gradient loops with matrix multiplication and transpose, achieving a 200–300x speedup and about 27% accuracy on 10,000 test images.

Raising the learning rate to 0.1 increases accuracy to about 91% on 28x28 handwritten digits. Visualize weights and explain how bias and the activation function influence digit classification.

Increase hidden neurons or add hidden layers to improve neural network accuracy, then tune iterations and learning rate, using digit recognition as a case study.

Compare Python-based neural networks to Mathematica’s pre-trained nets for digit recognition; explore architecture changes, fast training, and accuracy improvements to around 98.5%.

discover how neural networks from scratch apply interpolation to solve tasks from simple addition to handwritten digit recognition, showcasing hands-on learning and practical machine learning skills.

Install Python and editors with Anaconda's individual edition and Python 3.9, then launch Anaconda Navigator to start Jupyter Notebook, explore notebooks, kernels, and running cells with shift+enter.